When enabled, this will offload the reward model weights to CPU when not in use. This can be useful when training on a GPU with limited memory. The weights will be moved back to the GPU when needed.